Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries.

OBJECTIVE: The COVID-19 pandemic increased the demand for rapid evaluation of innovation in health and social care. Assessment of rapid methodologies is lacking although challenges in ensuring rigour and effective use of resources are known. We mapped reports of rapid evaluations of health and social care innovations, categorised different approaches to rapid evaluation, explored comparative benefits of rapid evaluation, and identified knowledge gaps. DESIGN: Scoping review. DATA SOURCES: MEDLINE, EMBASE and Health Management Information Consortium (HMIC) databases were searched through 13 September 2022. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included publications reporting primary research or methods for rapid evaluation of interventions or services in health and social care in high-income countries. DATA EXTRACTION AND SYNTHESIS: Two reviewers developed and piloted a data extraction form. One reviewer extracted data, a second reviewer checked 10% of the studies; disagreements and uncertainty were resolved through consensus. We used narrative synthesis to map different approaches to conducting rapid evaluation. RESULTS: We identified 16 759 records and included 162 which met inclusion criteria.We identified four main approaches for rapid evaluation: (1) Using methodology designed specifically for rapid evaluation; (2) Increasing rapidity by doing less or using less time-intensive methodology; (3) Using alternative technologies and/or data to increase speed of existing evaluation method; (4) Adapting part of non-rapid evaluation.The COVID-19 pandemic resulted in an increase in publications and some limited changes in identified methods. We found little research comparing rapid and non-rapid evaluation. CONCLUSIONS: We found a lack of clarity about what 'rapid evaluation' means but identified some useful preliminary categories. There is a need for clarity and consistency about what constitutes rapid evaluation; consistent terminology in reporting evaluations as rapid; development of specific methodologies for making evaluation more rapid; and assessment of advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.

The accuracy of pulse oximetry in measuring oxygen saturation by levels of skin pigmentation: a systematic review and meta-analysis.

BACKGROUND: During the COVID-19 pandemic, there have been concerns regarding potential bias in pulse oximetry measurements for people with high levels of skin pigmentation. We systematically reviewed the effects of skin pigmentation on the accuracy of oxygen saturation measurement by pulse oximetry (SpO2) compared with the gold standard SaO2 measured by CO-oximetry. METHODS: We searched Ovid MEDLINE, Ovid Embase, EBSCO CINAHL, ClinicalTrials.gov, and WHO International Clinical Trials Registry Platform (up to December 2021) for studies with SpO2-SaO2 comparisons and measuring the impact of skin pigmentation or ethnicity on pulse oximetry accuracy. We performed meta-analyses for mean bias (the primary outcome in this review) and its standard deviations (SDs) across studies included for each subgroup of skin pigmentation and ethnicity and used these pooled mean biases and SDs to calculate accuracy root-mean-square (Arms) and 95% limits of agreement. The review was registered with the Open Science Framework ( https://osf.io/gm7ty ). RESULTS: We included 32 studies (6505 participants): 15 measured skin pigmentation and 22 referred to ethnicity. Compared with standard SaO2 measurement, pulse oximetry probably overestimates oxygen saturation in people with the high level of skin pigmentation (pooled mean bias 1.11%; 95% confidence interval 0.29 to 1.93%) and people described as Black/African American (1.52%; 0.95 to 2.09%) (moderate- and low-certainty evidence). The bias of pulse oximetry measurements for people with other levels of skin pigmentation or those from other ethnic groups is either more uncertain or suggests no overestimation. Whilst the extent of mean bias is small or negligible for all subgroups evaluated, the associated imprecision is unacceptably large (pooled SDs > 1%). When the extent of measurement bias and precision is considered jointly, pulse oximetry measurements for all the subgroups appear acceptably accurate (with Arms < 4%). CONCLUSIONS: Pulse oximetry may overestimate oxygen saturation in people with high levels of skin pigmentation and people whose ethnicity is reported as Black/African American, compared with SaO2. The extent of overestimation may be small in hospital settings but unknown in community settings. REVIEW PROTOCOL REGISTRATION: https://osf.io/gm7ty.